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Abstract 

The most studied aspect of statistical network models is their degree 
structure, reflecting the propensity of nodes within a network to form 
connections with other nodes. Yet many simple random graph models 
are understood only asymptotically; for finitely many nodes, they do not 
yield either closed-form statistical likelihoods or precise forward generat- 
ing mechanisms. In contrast, we provide exact statistical results, limit 
theorems, and large-sample approximations that govern the behavior of 
networks based on random weights whose pairwise products parameterize 
independent Bernoulli trials. For power-law degree sequences we make 
the important observation that the frequently observed exponential cutoff 
behavior can be explained as an effect of this sampling. This enables us 
for the first time to understand, from a statistical perspective, the het- 
erogeneity of network degrees observed in practice, and to explore how 
properties of degree sequences scale with network size. Our results thus 
provide new tools to quantify degree variation within and across network 
populations. 
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Network models play many important roles across the sciences, and have 
recently given rise to new tools for statistical inference [TJ [5] in fields ranging 
from biology [3] to computational social science 0]. The single most studied 
aspect of network models is their degree structure, reflecting the propensity of 
nodes within a network to form connections with other nodes. At the same 
time, scaling is a fundamental property of physical processes, including those 
that give rise to networks [5]. As we collect larger and more complex network 
datasets, the scaling properties of network degree sequences must therefore be 
quantified and understood, to better enable the reproduction of characteristics 
seen in these data. 
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Thus we must develop a precise statistical understanding of the notion of 
a degree distribution. To do so we work with a hierarchical network model 
construction that encompasses many known random graph models [SI [71 [5], while 
at the same time allows for new types of heterogeneous structure and likelihood- 
based statistical inference. Network degree structure is regulated both by scaling 
and by the mechanism responsible for network generation. Starting with a 
model that is fully determined by its degree sequence, we parameterize the 
probabilities governing these degrees in order to obtain a forward generating 
mechanism. This fully stochastic model is useful and important because it 
enables a heterogeneous population of networks to be realized through a single, 
self-consistent framework. 

1 Modeling the Degree Sequence of a Network 
1.1 Existing Models for Network Degrees 

The preeminent summary statistic of an observed network is its degree sequence. 
A natural modeling approach is to start by either fixing or sampling a lcngth-n 
vector d of positive integer degrees [6], and then to select uniformly at random 
from the set of all possible graphs on n nodes that admit d as their degree 
sequence. In practice, edges are sequentially assigned to nodes in ways that 
respect d akin to sampling with replacement [9]. Asymptotically as n — > oo 
this practical construction is valid, but it can result in graphs with loops and 
multiple edges [10] . 

An important modification relaxes the requirement of a specified d, replac- 
ing it with a set of n parameters describing the propensity of each node to form 
connections. Edges are then independent Bernoulli trials with success probabil- 
ities Pij for all 1 < i < j < n. Chung & Lu [TT] associated a nonnegative weight 
Wi to each node i and set p^ = WiWj /\\w\\ i, where ||io||i = X>fc=i w k- Under the 
normalization constraint that wf < \\w\\i for all i, it follows that the expected 
degree 'E(di) of the ith node is equal to Wi, and thus the unnormalized weights 
wi, . . . w„ can be interpreted as expected degrees. This model is immediately 
recognizable as a log-linear model logpij = log Wi + \ogWj — log ||w|ji, connect- 
ing to existing statistical methods such as maximum likelihood estimation of p^ 
from an observed network [2j [12] . 

The given expected degrees model of [TT] can be thought of as a special case of 
assigning edge probabilities using a kernel [5] Section 16.4]; see also [T5I ITT] , 
where limiting properties of the resultant networks are derived. Further sta- 
tistical insight was given by Chattcrjcc, Diaconis & Sly |15) . who (among oth- 
ers 0H]) observed that under a similar model with log[pij/(l —pij)] = w- t +u'j, 
the network degrees arc a sufficient statistic for estimation of w — meaning that 
a network appears in the model's statistical likelihood function only through 
its degrees. This model has the advantage of assigning equal likelihood to all 
graphs with the same degree sequence, though it loses the multiplicative struc- 
ture of [TJJ. Janson [T3] gives conditions under which these two models are 
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asymptotically equivalent as n — > oo, and Perry & Wolfe [T^] give graph spar- 
sity conditions for fixed n under which optimal likelihood-based estimates for 
both models can be obtained directly as pij = djdj/||d||i — a natural estima- 
tor based on Chung & Lu's formulation [TT], and for which wc develop a limit 
theorem below. This degree-based estimator already sees wide use in practice, 
most predominantly in the calculation of modularity statistics for community 
detection in complex networks [1] I16j . 

1.2 Parameterizing the Poisson— Binomial Degree Distri- 
bution 

Simple n-node graphs are represented by a symmetric adjacency matrix A: 

Aij = Bernoulli^), 1 < i < j < n; A Ti = A vjl A vi = 0. (1) 

Each network degree is a sum di =Y] j-^Ajj of n— 1 independent Bernoulli (py) 
variatcs, with pji — pij, and thus takes the Poisson-Binomial distribution on 
values k € {0, 1, . . . , n — 1}: 

p(d* = * i {p« = e f n ( n ( j - )) ■ ( 2 ) 

Here the ( n ~^ 1 ) sets ^4' index, for fixed i, all distinct ways of obtaining k suc- 
cesses out of the n — 1 Bernoulli trials {Aij}j^a, and each ^4' is the respective 
complement in {0, 1, . . . , n — 1}. Setting ptj — p for all 1 < i < j < ?i recov- 
ers the canonical random graph model studied by Erdos, Rcnyi, and others, in 
which case (J2|) reduces to the Binomial(n — l,p) distribution for all i. 

To fully understand degree distributions from a statistical point of view, we 
must be able to obtain a precise specification of degree behavior for any fixed 
network size n, as well as to develop large-sample properties of sequences of 
networks as n increases. For this reason we adopt the multiplicative model 
of [TT], but with a parameterization that enables individual constraints to be 
specified for a normalized weight vector 7r: 

Pij = Hi TTj , 1 < i < j < n; m G II„ C [0, 1], i = l,...n. (3) 

Note that relative to the parameterization p^ = WiWj /\\w\ \ i, the formulation 
of ([3]) decouples the edge generation probabilities , so that each depends only 
on two parameters, and dispenses with the need to constrain wf < \\w\\i for 
all i. This simple reformulation has important consequences for our statistical 
understanding of network models for fixed and growing n, and will later enable 
us to introduce a hierarchical model where the normalized weights ~K\ , . . . 7r n are 
considered as a set of statistically independent random variables. 

Thus far we have established that, conditional upon a weight vector 7r, each 
network degree di follows a Poisson-Binomial distribution F(di = k \ 7r) specified 
by (J2])- To understand the scaling properties of network degrees as n — > oo, we 
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will allow the range IT„ of each normalized weight in © to depend on n. Fixing 
n„ = [0,1] for all n ensures that elements of n can remain order one, while 
letting |II„| shrink with n will reduce the order of each pij as n — s- oo. 

To understand the effects of this n-dependent scaling, first observe that 
together ([lJ-Q imply, via direct calculation, the following mean, variance, and 
covariance expressions for network degrees: 

E(d t | 7T) = HA 3 I 7T) = TTiJ2 *i ' 

Vax(di | tt) = Var(A„- | tt) = E(d t | tt) - tt? ]T tt| , (5) 

Cov(d 4 , | 7r) = Var(A y - | 7r) = 77,^(1 - 7t 4 7Tj), i^j. (6) 

The average behavior of degree di in n thus depends on its normalized weight 7r^ 
scaled by the sum 1 1 tt \ | i , while its variance depends on the scaling | tt 1 1 i — txi \ \ tt \ \ \ . 
The covariance between distinct degrees, on the other hand, cannot exceed 1/4, 
and so the correlation between any two degrees in a single network realization 
will decay toward zero whenever Var(di | 7r) is growing in n for each i. This result 
is well known for various random graph models in the literature; the interplay 
among (j4j-(j6]) shows explicitly why and how rapidly (in n) this decorrelation 
occurs, as a function of our assumptions on the growth of ||7r||i in n. 

1.3 Degree Distribution Variability: Power Laws and Net- 
work Scaling 

For any fixed n and weight vector 7r, the methods above precisely quantify degree 
variability, in accordance with the Poisson-Binomial distribution P(d,; = k \ 7r) 
of ([2]) . The scaling of network behavior in n is equally critical to understanding 
the variability of the resulting network degrees. In general 1 1 "tt 1 1 § an d IWWi will 
grow at different rates in n, and their ratio will impact not only the degree of 
heterogeneity incurred from Bernoulli sampling under the likelihood specified 
by (TTJ) and the multiplicative model of but also important aspects of sta- 
tistical data analysis, such as our ability to estimate tt from a single observed 
network under this combined model. 

A natural example is given by graphs having power laws as limiting degree 
distributions, as illustrated in Fig. [T] below. Here the expected proportion of 
degree-fc nodes in a single network realization scales as k~@ for some power /3 > 
0. Our first theorem treats power-law networks corresponding to those studied 
in \17\ - It characterizes their degree sequences under the parameterization of ([3]), 
both when n is large and fixed, and asymptotically as n — > oo. 

To interpret this theorem, we need the following notation. Given two se- 
quences /„ and <?„, the notation /„ = uj(g n ) means that f n grows in n strictly 
faster than g n , so that eventually (in n) the ratio \g n /fn\ becomes arbitrarily 
close to zero. In contrast, /„ = 0{g n ) means that f n grows no faster in n 
than g n , in that \ f n /g n \ is eventually upper bounded by a constant as n — > oo. 
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Finally, /„ = o(g n ) means that f n grows strictly slower in n than <?„, so that 
\fn/gn\ eventually becomes arbitrarily close to zero. 



Theorem 1 (Power law degrees). Fix an exponent 7 £ (0, 1) and set 

7T, = 6>„i" 7 , i = 1,2, ...,n, 

m£/i 0„ an n-dependent sequence of scaling constants satisfying 6 n < 1 for all 
n, so that YL n = [0,9 n ] in the model of 

Then the expected value of the ith degree di \ ir is given by 



!(d i |7r)=^i-T [(l- 7 )- 1 n 1 -T + 0(l)] 



(7) 



and as n — > 00, each di \ ir approaches a Poisson random variable with mean 
E(di I ir). In particular, the sum of all n differences 



P(d, = k]ir) 



l(d i \ir) k e- E( - d >^'> 



k = 0, 1, . . . , n — 1, 



converges to zero, and thus so does each individual difference in turn. 

Whenever ~E(di \ ir) is growing in n as determined by ([7]), then di becomes 
close to its expectation in that di/ E(dj | ir) converges in probability to 1, and 
{di — K(di I 7r)) / ^K(di I 7r) converges in distribution to a Normal(0, 1) random 
variable. 

Finally, assume 7 S (0,1/2) and 6 n = w(\/log n/n 1 / 2-7 ), m which case 
each degree di will grow in n. In expectation, an observed degree dj from a 
single network realization, with j sampled uniformly from {1, . . . , n}, will obey 
for P = 1 + I/7 the power law 



»(dj > x) 



l + O 



o 



k E(dn|w) 

ratt a = 1 + o(l), /or sc e {E(d„ \ ir), E(d n -i \ n), . . . , E(d 2 \ ir)} 



1 

,2(i 



Proof. Approximating ||tt||i = dninT" 1 Y]7— 1 (i/n)~' y by a integral squeezing ar- 
gument gives ||7r||i = Q n [n 1_7 /(l — 7) + 0(1)] for all 7 £ (0,1); combining 
with ((J]) yields the expression for E(di | ir). 

In the same manner, we may verify that lim„_ ! . 00 [1 — Var(d^ | ir) / E(d, | ir)] = 
0. For every n = 1,2,..., this quantity controls the total variation distance 
between the Poisson-Binomial distribution of @ and a Poisson distribution 
with mean E(dj | ir) |18j . which is equivalent to the theorem statement. 

Next, observe that if E(dj | 7r) is growing in n, it follows from the limit above 
that \im n ^ 00 Vai(di\ir)/E(di\ir) 2 = 0, implying via Chebyshev's inequality 
that di/E(di\ir) converges in probability to 1. Growth of E(dj | ir) implies 
growth of Vax(di \ ir) — a condition sufficient for the Lindebcrg-Feller central 
limit theorem to apply. 
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Figure 1: Simulated degrees illustrating within- and between- network degree 
variability for a power-law network with exponent /3 = 3. The left-hand panel 
orders the degrees by number i, while the right-hand panel orders the degrees 
by size. 



For the empirical distribution of n observed degrees, we write F n (x, d \ 7r) = 
1 — nT 1 Yl2=i " (^» > x I 7r )' w ith the indicator function I(-) taking values or 
1 accordingly. We may then compute, for x in the given range of the theo- 
rem, E (n _1 ^Zr=i ^ (di > x \ 7r)J = n _1 ^"=1 "* (^« - > x I 7r ) as follows. For those 
i such that |x — E(d, | vr ) | > 2^/E(dj | 7r) Inn, the components P (dj > x | 7r) in 
this sum can be treated using Chernoff bounds for tail probabilities of Poisson- 
Binomial variates. Contributions from the remaining values of i become in- 
creasingly negligible as n grows large, and thus can be bounded. Collect- 
ing the corresponding error terms together then yields the stated result for 
n- 1 J2i= 1 V(d i >x\iv). □ 

In addition to verifying power law behavior under the network adjacency 
model Aij = Bernoulli(7r i 7r :( ), Theorem Q] reveals that the notion of a degree 
distribution has two distinct interpretations that can lead to radically different 
assessments of degree behavior. Figure Q] illustrates this by way of network 
realizations for the case f3 — 3, corresponding to the well-studied preferential 
attachment network model of [5] . The first result of Theorem [T] confirms the 
between-network variability of the ith degree as becoming Poissonian across an 
ensemble of network realizations; we can visualize this ensemble via the left- 
hand panel of Fig. [I] by looking across these realizations for any fixed i. Here 
the Bernoulli trials governing edge formation are the only source of variation. 
By contrast, the final result of Theorem [T] (which can be directly extended to 
the case /3 = 3) describes the expected within-network degree variability in 
terms of a power law. Here we see variability induced by the range of values 
of TTi, emphasized by the ordering of degrees within each realization as in the 
right-hand panel of Fig. [TJ 
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In the special case of an Erdos-Renyi graph, the above interpretations agree. 
Here all 7^ are equal rather than decreasing in i, and thus Theorem Q] as writ- 
ten does not apply. Uniquely in this special case, the degrees are identically 
distributed. Their correlation is furthermore equal to l/(n — 1), and thus 
as n becomes large, they decorrelate and behave like a random sample from 
F(di = k I 7r) — equivalently, like an ensemble of independent and identically dis- 
tributed degree realizations. Later we will see that a similar construction can 
be used to produce a wide range of desired degree behavior, by letting elements 
of 7r themselves comprise a random sample. 

More generally, as long as each Var(c?i | 7r) is growing in n, then all degrees 
within a single network decorrelate in n, allowing us to view a single realized 
degree sequence as representing an ensemble of essentially independent — though 
not identically distributed — degree realizations. This view of a network's set of 
realized degrees as a distribution prevails in the complex networks literature: 
it is a natural and meaningful empirical summary of within-network degree 
variability that respects the lack of any given ordering of the degrees. 

In this view we consider a set of observed or expected degrees associated 
with a single network realization, and summarize their empirical distribution 
as a histogram. Note however that the observed degrees are random, while the 
expected degrees are not. Thus when reasoning about observed network data, 
we must take between- network sampling variability into account. This is why 
we could not simply consider the distribution function of the expected degrees, 
F n (x, E(d I 7r)), in the proof of Theorem [T] and invert F n to solve E(dj | 7r) = 
[1 — F„]~ 1 (i/n) in the limit of large n. Doing so verifies that when degree 
index j is sampled uniformly at random, P(E(dj | tv) > k) oc k^ 13 , whereas the 
arguments of Theorem [T] are required to ensure that the observed degrees from 
a single network, taken together as a group, will exhibit this same expected 
behavior. 

1.4 A Power Law Central Limit Theorem 

We now turn our attention to estimation of ir from an observed network. Re- 
call that in the power-law setting of Theorem [TJ whenever an expected degree 
E(dj I 7r) grows in n, the limiting Poisson distribution of di will tend toward 
a Normal distribution. As we show below, when n is large and the estimator 
7r = dj/-^/||d || 1 is used to estimate 7Tj |12) . then 7^ is approximately distributed 
as a Normal(7rj,7rj/||7r||i) random variable. 

Theorem 2 (Power law central limit theorem for 7r). Assume as in Theorem\T\ 
that each iTi = Qni -1 , with 7 € (0,1). Define 7Tj = dij \J\d\ \ as an estimator 
of TTi, and assume E(dj | tr) is growing in n. Then as n — > 00, the standardized 
variate (7^ — 7T^ ) / a/ 7Ti / 1 1 tt 1 1 1 converges in distribution to a Normal(0, 1) random 
variable. 

Proof. The growth of E(dj | ir) in n drives this result. Expand (7^— 7T,) / \pKi / 1 1 7T 1 1 1 
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d t -E(d t \n) | E(di|7r)-7riv/pyT 



E(rf, | tt) 

TTjjklll 



We must show that the first ratio above converges in distribution to a standard 
Normal, the second ratio converges in probability to zero, and the final multi- 
plicative term to one; Slutsky's theorem then implies the result. To deduce the 
first result we use the Lindebcrg-Feller central limit theorem as in Theorem [TJ 
The second result follows from the delta method and Chebyshev's inequality. 
The third result follows by applying Chebyshev's inequality to i/||t||i to 
deduce convergence in probability to 1, and then appealing to the continuous 
mapping theorem to obtain the same result for 1^/ 1 1 <i| 1 1 / 1 1 tt 1 1 1 . □ 

If we restrict 7 € (0, 1/2) and n — w(v / log~n/ri 1 / 2 ~ 7 ) as in Theorem[T] then 
all degrees will grow in n, and consequently this central limit theorem will hold 
for all elements of 7r = 1 ^ 2 d. 

2 A Hierarchical Model for Network Popula- 
tions 

2.1 From Conditional to Marginal Degree Distributions 

An important shortcoming of the Poisson-Binomial model P(dj = k \ tt) is that 
it cannot give rise to a heterogenous population of networks, but instead is lim- 
ited to random network realizations from a fixed and pre-specified vector 7r of 
normalized weights. The simplest construction giving rise to population hetero- 
geneity will allow 7r itself to be random, resulting in a hierarchical generative 
model and a marginal degree distribution P(d; = k) = fP(di = k | 7r)/(7r) dix 
for some probability distribution /(tt). If we further assume each 7Tj to be inde- 
pendent and identically distributed according to a probability distribution /(tt), 
then each di will be identically distributed as follows. 

Proposition 1 (Hierarchical random graph properties). Let tti, . . .tt„ be a ran- 
dom sample from some probability distribution f(n) on [0, 1], and let d be the de- 
grees from an n-node simple graph whose edges are Bcrnoulli(7Ti7rj) trials. Then 
every element of d is identically distributed, taking values k € {0, 1, . . . ,n — 1} 
as follows: 



P{d = k) 




(8) 



Var(d) 



E(d) 



(n - 1)E(tt) 2 , 
(n-l)E(d) Var(7r) + 




(10) 



(9) 



Gov(di,dj) 



E(d) 



3(n-2) 



Var(?r) + 



1 -E(tt 2 )" 



(11) 



n — 1 



ri — l 
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Proof. Observe that ^ is invariant to permutation of its parameters, and thus 



E„ [Fid, =fc|7r)] = En (li; TTi 7Tj J (n^fc+i^yiv 

any i, the marginal expression in (J5J then follows by evaluating scalar expecta- 
tions with respect to each Ttj j^. The moments of (f9^- (fTTj) follow by marginal- 
izing their conditional counterparts in (U])-©, using the law of total covariance 
in the latter two cases. □ 
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2.2 Reproducing Properties of Network Degrees 

To understand the properties of observed network degrees under the hierar- 
chical model described by Proposition [T] we must determine how our choice 
of f(n) interacts with the sampling giving rise to d, as characterized by the 
Poisson-Binomial distribution of ([2]) via (jHJ. For well-behaved densities /(tt), 
the following theorem characterizes this interaction. 

Theorem 3 (Reproducing property). Assume /(7r) is bounded and everywhere 
nonzero on [0,1], with three bounded derivatives. Then 



n 



!(7r).P(d=fe) = /(miii[l,^ y ])[l-F(fc;n,E(7r))] + f(n) ) 



where F (k; n, E(7r)) is the cumulative distribution function of a Binomial(n,, E(7r)) 
random variable. The error term £(n) is 



£(n) 



0{$=fa), 0<k<nE(n); 
k O(^=;)/(l)[l-F(fc;n,E(7r))], nE(ir) < k < n. 



If f(n) = Uniform(0, 1), then £{n) is identically zero and 

nE(7r) -P(d = k) = 1 - F(k;n,E(w)). (12) 

Proof. We prove (jT2"j) and outline the general case for f(n) satisfying the theorem 
hypotheses. Replace E(ir)ir by it in ([5]) to obtain 

this is the form of a mixed binomial [19], except for the appearance of E(ir) 
as the upper limit of integration. If f(n) = 1 we immediately obtain (|12[) , 
as in this case the integral of (|T3|) evaluates to the product of the regularized 
incomplete Beta function I m r^(k + l,n — k) = 1 — F(k; n, E(7r)) and the Beta 
function /3(k+l, n—k). 

For the general case we proceed for each k via a Taylor series expansion of 
/(•7r) about min[l, /c/(nE(7r))], splitting the range of integration of (fTB"]) for each 
tail k tiE(7t) according to \k— nE(7r)| = uj(y/n) or \k — nE(7r)| = (9(-y/n). 
For each Taylor term, the integrals over respective regions evaluate to differ- 
ences of incomplete Beta functions. By choosing the regions as indicated, we 



can use the boundedness of f(n) and Hoeffding's inequality F(k; n,E(7r)) < 
0.5 exp(— 2(nE(7r) — k) 2 /n) for k < ?iE(7r) to show that most of these terms 
converge exponentially in n to or a nonzero constant. 

We are then left with the main term of (|12[) plus corresponding error terms, 
additive in the left tail (when k < nE(7r)) and multiplicative in the right (where 
nE(7r) F(d = k) is going rapidly to zero). Repeated application of this argument, 
combined with Lagrange's form of the Taylor remainder and the boundedness 
of the first three derivatives of f(n), yields £(n) as stated. □ 

Thus Thcorcm|3]fully characterizes degrees from independent Bcrnoulli(7rj7Tj) 
edges whenever the normalized weights ~K\ , . . . 7r„ comprise a random sample 
from some smooth and bounded density /(7r), nonzero on the entire unit inter- 
val [0, 1]. In particular, we see directly that any such f(jr) is reproduced in its 
entirety on the interval < k < nE(ir). 

This result also confirms that the marginal degree distribution P(d — k) re- 
sults from the multiplicative interaction of /(7r) with the function 1— F(k; n, E(7r)), 
which is strictly decreasing in n and modulates /(min[l, fe/(nE(7r))]) over the 
entire range < k < n — 1 of d. As n grows large, the scaled probability mass 
nE(-7r) ¥(d = k) is asymptotically equal to the product of these terms, the latter 
of which is the survival function of a Binomial(n, E(7r)) random variable; i.e., the 
probability that such a random variable exceeds k. When \k — nE(n)\ = uj{y/n), 
this probability tends exponentially quickly to 1 or by the arguments in the 
proof above, whereas in the transition region \k — nE(7r)| = 0(^/n) we obtain 
a scaling between and 1, attaining 1/2 as k approaches nE(7r). The leftmost 
panels of Fig. [2] illustrate this behavior when f(ir) = Uniform(0, 1) in accordance 
with the exact expression of (|12[) . with the remaining panels exemplifying the 
behavior of other smooth densities as indicated. 

2.3 Networks from Scale-Free Distributions 

An important example to which Theorem [3] does not directly apply is that of 
a scale- free or power-law density f(ir) oc tt~^, as this density is unbounded 
near zero. However, just as Theorem [T] demonstrated that fixing 7^ = i 1 
for 7 G (0, 1/2) led asymptotically to the power law F(d = k) cx fc -/S , where 
= 1 + I/7, we show that such behavior is also achievable via a reproducing 
property for power-law densities. 

Theorem 4 (Power law reproducing property). Fix some /3 > ; and let 
7Ti,...7r n be drawn independently from f(ir) oc Tr~@ , restricted to the range 
[I, u] C [0, 1], with I > whenever (3 > 1. Then for all k G {0, 1, . . . ,n — 1} such 
that k > (3 - 1, 

nE(7r) • P(d = k) = f( (n+ % M ) [W)( fc + l-P,n-k) 
-/ 1KW (Hl-ftn-fe)] 



1 + n ( ( "~ fc) ] 
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/(7r)=Uniform(0,l) /(7r)=Beta(2,3) /(7r)=Beta(2,l) 




500 500 500 

k k k 



Figure 2: Simulated empirical degree distributions illustrating the reproducing 
property of Theorem [31 whereby f(ir) (top row) is reproduced for P(d = k) on 
the interval < k < nE(7r) (bottom row; n = 500). Note the right tail behavior 
apparent in the bottom row, due to the multiplicative effect of 1 — F(k; n, E(7r)). 

Proof. Let c _1 = J," n" 13 dn, and note that when (3 > 1, we must have I > 
to ensure integrability. We substitute /(tt/ E(tt)) = c (tt/ E(tt))-' 3 in (TT3j) , 
adjusting the limits of integration to ZE(-7r) and uK(jr), respectively. For k > 
(3 — 1, we may then write the result as the difference of two incomplete Beta 
function integrals, yielding 

w d = k ) = r(n + i) r(fc + i-/3) 

1 ; nE( 7 r) 1 -^r(n+l-/3) T(fc + 1) 

• [4 EW (fc + l-j8,n-fc)- J IEW (fe + l-/3,n-*)] . (14) 

From the standard Taylor series for lnr(z), it is easily verified that T(z)/T(z — 
/3) = Z P[l - {(3 /2){l + (3){z~ (3)- 1 + 0({z - P)~ 2 )]. Applying this to (H), first 
with 2 = n + 1, and then with z = k + 1 via the Taylor series of T(z — p)/T(z), 
yields the stated result. □ 

Theorem [4] shows that the scale-free density f(ir) oc 7r - ' 3 leads to a power- 
law decay in observed degree sequences across a range of k. For k < P — 
1, it can be verified directly that the probability of observing d = k decays 
exponentially in n. Similarly, when k > wE(7r)n, an exponential tail decay due 
to I u ) (fc + 1 — P,n — k) will rapidly take over. In fact, by analogy to the proof 
of Theorem[3j the difference I u e(tt) {k + 1 — P, n — k) — IiE(Tt){k + 1 — P, n— k) goes 
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Figure 3: Left panel: Averaged degrees from power-law networks on n = 1000 
nodes, generated from f(jr) cx 7r~ 3 on [1/3,1], with E(7r) = 1/2. Rapid decays 
are visible near k = 500/3 = iE(?r)n and k = 500 = uE(tt)ti. Right panel: a 
log-log plot of the transition region near \n(uM(ir)n) ~ 6.2, showing the exact 
and empirical distributions along with a Taylor expansion of I u -e(tt) (fc+l — n—k), 
corresponding to an exponential cutoff effect. 

exponentially quickly to or 1, with transition regions for |ZE(7r)n— k\ = 0{^/n) 
and |uE(7i>-fc| =0( v / ^). 

Investigating this behavior in more detail reveals that the effects of Bernoulli 
sampling can explain power laws with exponential cutoff, which in practice often 
provide a better fit to data [3U] . Power laws are typically identified from data by 
taking logarithms of empirical frequencies approximating ¥(d = k), and identi- 
fying a linear trend. Theorem [4] shows that lnP(d = k) decomposes as a sum of 
contributions from In [/((fc+l)/[(n + l) E(7r)])] and In [/„£(,) (k+1 — (3, n— k)\ . 
The latter term will serve to rapidly reduce probability mass as k exceeds 
uK(ir)n, and it is straightforward to show that a first-order Taylor expansion of 
this term corresponds to an exponential cutoff effect. 

Figure [3] illustrates the consequence of this sampling-induced decay. The left 
panel shows P(d = k) and its empirical estimate for the case j3 = 3, with the 
right panel showing the transition region near nE(7r)w on a logarithmic scale, 
along with a first-order Taylor expansion about the center of this region. This 
sounds a cautionary note to practitioners, in that effects ascribed to a model of 
degree sequence behavior may in fact be due purely to the effects of sampling. 

3 Scaling and Sequences of Networks 
3.1 Asymptotic Scaling Regimes 

Theorems [3] and 2] confirm that marginal properties of degrees from independent 
Bernoulli(7Ti7rj) edges are determined by an interaction between a chosen dis- 
tribution f(ir), and the Poisson-Binomial sampling distribution of ([2]). To this 
end, Fig.|4]shows four different distributions /(7r), each with mean E(7r) = 1/2, 
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/(tt) = Pareto(l/3,l:/3=3) /(tt) = Unifomi(0,l) 




200 400 600 800 1 000 200 400 600 800 1 000 

Sorted degree 



Figure 4: Estimated values pij = cWj/||d|| i of pij = iriirj, sorted from largest 
to smallest degree to emphasize decay. Estimates are averaged over 10 network 
realizations for each indicated distribution /(tt), with E(ir) = 1/2 in each case. 

in terms of estimates pij = djdj/||d||i, sorted from largest to smallest degree. 
The first corresponds to the scale-free example f(ir) oc 7r~ 3 considered in Fig. [3] 
and Theorem |4j the second to the Uniform(0, 1) example of Theorem [3l the 
third to a U-shaped distribution Beta(l/2, 1/2); and the fourth an Erdos-Renyi 
graph with p^ = 1/2. These distributions exemplify the wealth of structure 
that can be reproduced in this modeling regime. A key feature is that, despite 
their being standardized to have the same expectation, the resulting degrees all 
still retain distinctive structural properties. 

This example motivates the question of precisely what ranges of limiting 
degree distributions arc achievable by hierarchical simple random graph mod- 
els. As long as elements of 7r remain order one, the combined effects of the 
reproducing property and Bernoulli sampling, as characterized by Theorems [3] 
and 21 will give rise to the behaviors above. Real networks exhibit substantial 
heterogeneity in their degree distributions, however, and so we must also under- 
stand modeling regimes in which these two effects are suitably balanced. This 
will enable us to construct degree distributions that scale in a controlled way 
with n, and exhibit greater heterogeneity than is possible when the order of tt 
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remains constant in n. For this reason, we introduce the notion of an asymptotic 
scaling regime, whose properties follow directly from (|9))- (|11[) in Proposition [TJ 

Proposition 2 (Asymptotic scaling regime). For n a random variable on [0, 1] 
with fixed mean E(-7r) and variance Var(7r), we define an asymptotic scaling 
regime by the linear map ^^(tt), representing an n-dependent scaling and shift- 
ing of 7r as follows: 

7r (n) (7r) = 9 n n + u n \ 9 n = v n = (15) 

n°<> n°" 

parameterized by (be, eg), (b u ,c u ), all strictly greater than zero. 

Let 6 m j n = mhY(b v ,be). Degrees d^ generated from ^^(ir) have mean and 
variance as follows, with E^- 71 ' = 0(n~ 2brain ): 

E(d (n) ) =n 1 - 2b ° (c e E(7r) + c u n b "- b ") 2 + £ {n} (16) 
Var(dW) =n 2 < 1 - 2 ^) (c e E(tt) + c u n b °- b ») 2 

• [c^Var( 7 r)+n 2!,fl - 1 ] + n(£ (n ^ 2 . (17) 

Note the dependence of E(d( n >) and Vai(d^) on n in (Tf7j]) and ((T7)) . re- 
spectively. Beyond the case of fixed weights scaled by 6 n , as considered in 
Theorem [TJ this n-dcpcndcncc allows us to quantify the basic properties of S n ^ 
in any asymptotic scaling regime, as we can shift and scale 7r independently via 
6 n and v n . We observe directly from ([TBI and (fT7|) that two regimes stand 
out as particularly important: the balanced asymptotic scaling regime in which 
be = b v , whereupon the sampling variation in balances with the variabil- 
ity in 7r; and the case of more limited heterogeneity in which & m i n = 1/2, so 
that both the mean and variance of each d^ n ' will converge to a finite limit in 
increasing n. 

The expressions in (fT6|) and (fT7|) also highlight that a natural rescaling of 
d( n ) i s given by l/n 1 ~ 2&min . This is reminiscent of the deterministic construc- 
tion of Theorem [TJ for which an appropriate rescaling is 1/n 1-27 for 7 < 1/2. 
In Proposition [5] the overall magnitudes of the degrees can grow in a fashion 
commensurate to Theorem [TJ but the degree variances will differ because tt is 
random. 

3.2 Networks with Controlled Growth of Expected De- 
grees 

We start by exploring asymptotic scaling regimes in which & m i n , the slowest of 
the shift and scale decay rates, lies in (0,1/2). In this regime, each expected 
degree grows with network size n, but the expected proportion of nodes that a 
single node will connect to, compared to the n— 1 possible, is decaying as n -2bmin . 
The total number of network edges will grow as ji 2 ( 1— ^niin) ; thus determining 
overall network density. 
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If b v — > oo in this regime, so that the shift term v n goes to zero, we will 
achieve a reproducing property akin to Theorem [3j This can be shown by 
adapting a similar Taylor series argument to the magnitude of the derivatives of 
f(ceir/n be ), with the error terms adjusted commensurately. As described above, 
the natural rescaling of each is then given by l/n 1_2f>s , so that the resultant 
random variables have order-one moments, yet inherit the properties of 7T. 

3.3 Networks with Restricted Heterogeneity in Limiting 
Degrees 

In certain networks it may be reasonable to expect that degree heterogeneity 
has saturated, meaning that (unlike in the case of controlled degree growth 
considered above) the distribution of degrees will not change appreciably as the 
network scales to larger sizes. This scenario implies a distribution of that 
is independent of n, and for which neither the mean nor the variance of each 
dW depends on n. 

From (fl6|) and (|17j) . we see that this effect is achievable if & m ;„ — 1/2. A 
classic example is given by an Erdos-Renyi graph in the limiting Poisson regime: 
starting with an arbitrary f(n) on [0, 1], if we let be — >• oo so that the effect of 
f(n) is negligible, then for b v = 1/2 we will obtain a Poisson degree distribution 
in the limit as n — > oo. This is because 7p"' will concentrate at c^/^/n, and 
hence eachp^ will scale as c 2 /n, so that the distribution of each S- n ^ approaches 
that of a Binomial(n — 1, c 2 /n) random variable. 

Importantly, we may also achieve non-Poisson limits in this regime. Observe 
that if 6 m i n = 1/2, then the dispersion Var(d("))/E(<f(")) is given to leading 
order by 1 + c 2 , Var(7r)/n 1-2be . This quantity converges to 1 when bg > 1/2, 
but when bg = 1/2, it tends to 1 + c 2 , Var(7r). Since the dispersion of a Poisson 
random variable is unity, we immediately see that when be = 1/2, the limiting 
distribution of d^ cannot be Poisson. 

The product c 2 , Var(7r) is important in this non-Poisson setting, as it charac- 
terizes the limiting over-dispersion of d^ relative to a Poisson variate, and will 
in turn govern limiting properties of the degrees. Note that c 2 , and Var(7r) can- 
not both be determined from a single network observation (though their product 
can); instead, to estimate them from data would require multiple realizations of 
the same network generating mechanism at different sample sizes n. 

3.4 An Over-Dispersed Degree Limit 

We conclude with a final example of degree heterogeneity that contrasts the 
Erdos-Renyi Poisson limit resulting from b v = 1/2 and bg — > oo. There n^ n ' 
behaved like c„/ ' yfn as n — > oo, and f(ir) had negligible effects. If we consider 
the opposite scenario of b u — > oo and be = 1/2, and take the entropy-maximizing 
Uniform(0, 1) distribution, then the variation in = cen / 1 \fn will give rise to 
the following over-dispersed limit. 



15 



Proposition 3 (An over-dispersed degree limit). Let f(ir) = 1 on [0,1], and 
define 7P n '(7r) via the asymptotic scaling regime of (|15[) . with b v — > oo and 
bg = 1/2. Then as n — > oo, each degree d^ n ' converges to a random variable 
such that for ( = c 2 g /2, we have E(d(°°)) = C/2, Var(d(°°') = C(C + 6)/12, 

and 

P(dM=*)=ih-£^Y * = 0,1,.... 

Proof. We start from ((HJ, which gives a general expression for the marginal 
probability P(d'") = fc). Two different approaches must be adopted: one for 
k = o(n), and the other for k oc n. In the former scenario, we evaluate the limit 
of the expression (1 — E(7r)7r)" _fc ~ 1 from (|SJ) using Um n _ ) . 00 (l— x/ri) n = e~ x , and 
evaluate the resulting integral directly. For k oc n, the probability of obtaining 
k decays exponentially in n. If we bound ([S| in this regime using Stirling's 
inequality, the triangle inequality can then be used to obtain convergence to the 
stated P(d(°°> = k) . The mean and variance of this limiting distribution are 
then calculated directly. □ 

Observe from the above that ti» is a uniform variate whose range [0,cg/ y^n] 
is shrinking toward zero at rate 1 / y/ri, without any significant shift parameter 
v n to offset the reduction in its mean. Thus, as we expect, Proposition[3]vcrifies 
that P(d(°°) = k) decays monotonically as k increases from zero. Yet d(°°) is over- 
dispersed relative to a Poisson variate, as Var(d( 00 ))/E(d( 00 )) = 1+ E(d(°°))/3. 
Since the dispersion of d^ depends linearly on its mean E(d^°°') = c|/4, it 
follows that for increasingly larger values of eg , we can expect to see increasingly 
variable realizations of d(°°) relative to a Poisson limiting regime. In this same 
manner, other asymptotic scaling regimes can yield different limiting behaviors. 



4 Discussion 



In this article we have introduced a hierarchical statistical model of simple ran- 
dom graphs, and used it to give a precise answer to the question, "What is a 
degree distribution?" We have developed exact statistical results, limit theo- 
rems, and large-sample approximations that govern the behavior of networks 
based on our construction. The advantage of using a statistical model to do so 
is that it links networks of varying sample sizes to a single range of parameters, 
and indicates under what assumptions these parameters can be estimated. 

To study a fully generative modeling framework, we placed a stochastic 
model on normalized weights giving rise to a network adjacency matrix. This en- 
abled us to understand the notions of distributions of degrees between networks, 
as well as within the context of a single network. We determined a reproduc- 
ing property associated with these normalized weights, and explicitly quantified 
effects for finite but large n — including explaining the exponential cutoff be- 
havior often noted in this setting as an effect of Bernoulli sampling. Finally, 
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to understand explicitly the effects of sampling variability, we explored vari- 
ous asymptotic scaling regimes. These permitted us to scale order-one weights 
toward zero as n — > oo. thereby allowing for the control of limiting degree dis- 
tributions across network sizes. 
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